While consumers may access media items, such as movies and television shows, by receiving over the air signals or by subscribing to a cable or satellite television provider, increasingly consumers are accessing content over Internet-based systems. Some Internet-based systems allow users to download or stream content over the Internet to a variety of client devices. For example, an Internet-based media system may provide content to users via a personal computer, a set-top box, or a personal mobile device, such as a smart phone or tablet computer. In particular, streaming media systems enable users to access media content in a stream, such that the users may begin consuming (e.g., watching and/or listening to) content before the entirety of the content is delivered to the user's client device. Such a system allows users to access content while avoiding a potentially lengthy download process before beginning to consume their selected content.
To provide users with a satisfying, high-quality content experience, content service providers may provide content recommendations to users. Recommendations for a particular user or set of users may be based on a variety of factors including the particular user's location, personal viewing history, type of device being used to view the content, and other factors. It is desirable that recommendations be as close to what the user actually wishes to view as possible.
Various analytical tools may be used to create a system that recommends desired content to specific users. One such tool is machine learning. Machine learning involves the formulation of complex models and functions that are able to be predictive. Machine learning may involve, for example, providing a computing system with a set of example inputs and desired outputs so that the system can attempt to derive a general correlation that maps those inputs to the desired outputs. Such inputs may be complex n-dimensional data objects. Inputs in such formats are typically referred to as feature vectors, or simply features.
The efficacy of machine-learning tools is highly dependent upon the type of data that is provided to the machine-learning processes. Thus, it is desirable to experiment with different types of feature vectors that may provide better results from a machine-learning model. It is also desirable to test the models that have been derived from the machine-learning process in a live environment. In other words, it is desirable to determine whether the information obtained from the machine-learning model provides better recommendation outcomes. It is desirable that experimenting with different types of inputs and performing live testing be performed in a time efficient and effective manner.
These drawings will be better understood by those of ordinary skill in the art by reference to the following detailed description.